Credit initialization in systems with proactive flow control

ABSTRACT

The inventive mechanism initializes the credit and debit registers used in a network computer system that uses proactive flow control. During initialization, the credit register is initialized to zero, while the debit register is initialized to the amount of queue space available in that particular chip release. Once the debit register is non-zero, it eventually releases the credits. These credits will be added to the credit register. These activities of adding and releasing credits take place during normal operation. Thus, the sender and destination nodes do not have to undergo any other initialization stages for setting up the credits.

BACKGROUND OF THE INVENTION

[0001] The communication between different nodes in a network computersystem occurs in a producer-consumer fashion. A sender or producer nodesends a transaction to one or more destination or consumer node(s). Eachdestination node stores the transaction in an internal queue and‘consumes’ it at a later point of time. For example, in a multiprocessorsystem, a snoop request by the source node is stored in an internalqueue by all the destination nodes. Each node will act on that snooprequest at some later point in time, and then removes the request fromits queue. Thus, a producer should not send a transaction to adestination node that does not have the space required to store thetransaction in its queue. The space requirement is ensured by theunderlying flow-control mechanism.

[0002] The flow-control mechanism may be reactive or proactive. In thereactive approach, the destination node tracks its own queues and sendsa queue_full signal to the respective sending nodes when the queues arefilling up. However, the proactive flow-control mechanism is consideredto be more useful in systems with a point-to-point interconnecttopology, for example cross-bar, ring, mesh, or hypercube. In theproactive approach, the sender node keeps track of the amount ofavailable queue space in the destination node through the amount of‘credits’ it has for the corresponding queue in the destination node.Each credit translates to a certain number of entries in the destinationqueue. The number of available credits is maintained in a register inthe sender node. This extra space register is referred to as no_credit.Each node will have one no_credit register for every queue in everydestination node to which the node can send a transaction. A sender doesnot send a transaction if the amount of space required by thetransaction is more than the number of credits it possesses for thequeue in the destination node.

[0003] The destination node ‘releases’ these credits to the sender, whenit ‘consumes’ a transaction from its queue. A destination node maychoose to release credits as it is unloading a transaction. Or thedestination node may release the credits after the transaction has beenunloaded. The amount of credits released by the destination nodecorresponds to the amount of space freed up in the queue by consumptionof a transaction. The destination node keeps track of the amount ofqueue space it will be releasing in a debit register. This register isreferred to as no_debit. The destination node releases the credits fromthe no_debit register when it gets an opportunity to send a transactionto the sender node that had sent the transaction(s) earlier. Note thatthe destination node may append the credits onto another transactionpassing through the sender node.

[0004] After releasing credits, the no_debit register is decremented bythe amount of credits released. On receipt of these credits, the senderadds the credits to the amount of credits available, no_credit, for thatdestination node. This way the sender keeps track of the amount of queuespace it can safely use in the destination node for forwardingtransactions.

[0005] Note that each node many have more than one queue associated withanother particular node. For example, each node may have a request queueand a response queue to store requests and responses from another node.Therefore, separate credits will be maintained for each queue by thenode.

[0006] A problem occurs with the initialization of credits in theno_credit and no_debit registers, either during an initial start up orafter a reset. Different components of the system may undergo differentdesign revisions and the corresponding queue sizes may change over timefor the same component. Thus, the sender nodes cannot be hardwired toassume the queue size in the destination nodes since that size maychange with changes made to the components of the destination node.

[0007] Therefore, during power up, the sender does not know how manycredits to allocate in the registers for each queue in the destinationnode. The prior art uses a software mechanism with additional hardwareto perform a credit initialization. Both the credit and debit registersare set to zero. The software is used to read the maximum number ofcredits for each destination node, and then write that into the sendernode. However, in order to perform this credit initializationtransaction credits must be available. Thus, this forms a paradox wherea credit initialization cannot take place until credits are available.To solve this problem, the prior art uses a ‘power on mode’ which allowsall transactions to be sent without using credits. This mode requiresextra logic and imposes a certain minimum queue sizing to allow for thereading and writing of registers. Thus, the prior art mechanism requiresthe user to have extensive knowledge of the system.

[0008] Therefore, there is a need in the art for a mechanism that allowsfor credit initialization without involving software intervention andusing additional hardware.

SUMMARY OF THE INVENTION

[0009] These and other objects, features and technical advantages areachieved by a system and method which employs the mechanisms already inplace and used for normal operations. During normal operations, if thesender has credits, then is may send transactions to the destinationnode. If the destination node has debits, then it releases them back tothe sender node. During initialization, the credit register, no_credit,is loaded with a zero. Thus, the sender node begins operations with nocredits, and cannot send any transactions. The debit register, no_debit,is loaded not with zero, but with the maximum credits representing thesize of its queue. Thus, the destination register begins operations withthe debit register full, and through normal operations, releases thecredits back to the sender node. Thus, the sender node will then havecredits and can then send transactions, and the destination node willhave an empty debit register. This results in faster boot times, sincethe initialization is completed as soon as the chips are out of theirreset state, which is 1 cycle after reset is withdrawn, because no timeis spent on software operations in a start-up mode.

[0010] It is a technical advantage of the invention to performinitialization of the queue registers through normal flow controloperations.

[0011] It is another technical advantage of the invention to eliminatethe need for software credit initialization.

[0012] It is a further technical advantage of the invention to eliminatethe need for additional hardware and registers for a power on mode.

[0013] It is a further technical advantage of the invention to havefaster boot times, since the initialization is completed as soon as thechips are out of their reset state or 1 cycle after reset is withdrawn.

[0014] It is a further technical advantage of the invention that theinventive mechanism will work with any sized queue and does not have anyminimum sizing requirements for initialization.

[0015] The foregoing has outlined rather broadly the features andtechnical advantages of the present invention in order that the detaileddescription of the invention that follows may be better understood.Additional features and advantages of the invention will be describedhereinafter which form the subject of the claims of the invention. Itshould be appreciated by those skilled in the art that the conceptionand the specific embodiment disclosed may be readily utilized as a basisfor modifying or designing other structures for carrying out the samepurposes of the present invention. It should also be realized by thoseskilled in the art that such equivalent constructions do not depart fromthe spirit and scope of the invention as set forth in the appendedclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] For a more complete understanding of the present invention, andthe advantages thereof, reference is now made to the followingdescriptions taken in conjunction with the accompanying drawings, inwhich:

[0017]FIG. 1 depicts a network system having two nodes interconnected bya bus;

[0018]FIG. 2 depicts the flow control mechanism used by the sending nodein transmitting a packet to the destination node;

[0019]FIG. 3 depicts the flow control mechanism used by the sending nodein receiving a packet from another node;

[0020]FIG. 4 depicts the flow control mechanism 40 used by thedestination node in receiving a packet from the sending node; and

[0021]FIG. 5 depicts the flow control mechanism in the destination

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022]FIG. 1 depicts a simple network system 10, having two nodes, nodeA 11 and node B 12 connected by bus 13. Each of the nodes will have atleast two queues, a request queue and a response queue, although eachnode could have more queues. By having separate request and responsequeues a deadlock situation is avoided. Node A 11 uses its request queuefor holding requests that are being sent to node B 12. Node B 12 usesits request queue for holding requests receiving from node A 11. Node A11 uses its response queue for holding response that are being sent tonode B 12. Node B 12 uses its response queue for holding responsesreceiving from node A 11. Note that for node A 11 to send a response,node A must have received a request from node B 12, however for purposesof simplicity, assume that node A 11 is the sending node, and node B 12is the destination node, although either node could be sending orreceiving.

[0023]FIG. 2 depicts the flow control mechanism 20 used by node A 11 insending a packet P to destination node B 12. The sending node generatesthe packet P 21 for a particular queue, queue i, which could be theresponse queue or the request queue, or another type of queue. Note thatthe packet P may have been generated elsewhere, and is merely passingthrough node A onto node B. The arbitration and forwarding logic 22 ofnode A determines how much queue space this particular packet P willconsume in queue i of node B. This space is called space_reqd_P. Thelogic then determines if the credit register for queue i of node B,no_credit_B_Qi, has a sufficient number of credits for the spacerequired for packet P. The logic determines 23 whether no_credit_B_Qi isgreater than or equal to space_reqd_P. If not, then node A 12 waits forcredits 24 to be released from node B 12 before sending packet P. Ifthere is enough credits, then packet P is scheduled 25 for delivery tonode B 12. The packet P could go to a dispatch queue or be sent out tonode B 12. The credit register is decremented 26 by the amount of spacefor packet P, no_credit_B_Qi is to no_credit_B_Qi minus space_reqd_P.Packet P is then sent out 27 to node B 12.

[0024]FIG. 3 depicts the flow control mechanism 30 used by node A 11 inreceiving a packet Q from another node 31. The arbitration andforwarding logic of node A determines 32 whether packet Q has anyreleased credits for node A 11. If packet Q contains credits, then thelogic increments the appropriate registers by the number of credits 33.For example, if packet Q was sent by node B, then the packet may containcredits for credit register no_credit_B_Qi, in which case,no_credit_B_Qi is incremented by the number of credits in the packet.After incrementing or if there were no credits for node A, the logicthen determines whether packet Q is destined 34 for node A 11. If not,the packet is forwarded to the appropriate node 35, which may be eitherthe destination node or the next node in the chain of nodes to thedestination node. If node A is the destination node, then the packet isforwarded 36 to the processing unit inside node A. Note that thissequence assumes that every node is only keeping track of the credits ofthe adjacent nodes. Further note that even if the packet is not destinedfor node A, an adjacent node may have appended credits into the packetfor node A. If credits were passed back to non-adjacent nodes on asystem where there is no store and forward, the credits would bereleased at the destination nodes only.

[0025]FIG. 4 depicts the flow control mechanism 40 used by node B 12 inreceiving a packet P 41 via bus C 13 from node A 11, as shown in box 27.Since node B is also a sending node, its arbitration and forwardinglogic determines 42 whether packet P has any released credits for node B12. If packet P contains credits, then the logic increments theappropriate counters by the number of credits 43. After or duringincrementing or if there were no credits for node B, then the logic thendetermines whether packet P is destined 44 for node B 12. If not, thepacket is forwarded to the appropriate node 45, which may be either thedestination node or the next node in the chain to the destination node.If node B is the destination node, then the packet is forwarded 46 tothe processing unit inside node B. Packet P then enters 47 queue i innode B 12. If packet P is a request packet, then queue i is the requestqueue. If packet P is a response packet, then queue i is the responsequeue. After a time period, packet P will leave queue i and be consumed48. For example, if packet P is a request, then as soon as the responseis generated the request is removed. Another example is that the packetmay be moved to another queue within node B. After the packet isconsumed, then credits are released to the debit counter, debit_A_queuei, which is incremented 49 by the appropriate value of the credits. NodeB will unload the credits in the debit register to node A whenappropriate 54. Note that a specific packet, a credit packet, could begenerated by node B to send the credits back to node A, or node B couldappend the credits onto a data or information packet that is eitherdestined node A or passing through node A. Note that node A does nothave to be the destination of the information packet. Further note thatnode B may not have constructed the information packet, node B may justbe passing the packet along the node chain. The entire contents of thedebit register may be loaded into the packet bound for node A, or only aportion of the contents may be loaded. Thus, either several packets or asingle packet may be used to send the credit back to node A. The debitregister, debit_A_queue i is decremented according to the number ofcredits unloaded. Note that a specific wire may be also used to send thecredits back instead of packets. The returned credits will be handled bynode A as shown in FIG. 3.

[0026]FIG. 5 depicts the flow control mechanism 50 in node B during apower on or reset condition of the system 10. The arbitration andforwarding logic of node B first determines whether the reset conditionhas been lifted 51. If not, node B waits until the reset condition islifted. After the reset condition has been lifted, the debit register,debit_A_queue i is loaded 52 with the maximum value that queue i canstore. The debit register is then unloaded and decremented using thenormal release mechanisms 53, as shown in FIG. 4. Note that the maximumvalue of queue i is set in hardware, e.g. via a register, since thevalues of the queues of are known at the time of design. Thus, nosoftware is needed.

[0027] The arbitration and forwarding logic of node A would alsodetermine whether the reset condition has been lifted, and if not, nodeA waits until the reset condition is lifted. After the reset conditionhas been lifted, the credit register, no_credit_B_Qi is loaded with thevalue zero. A packet will come from node B containing the releasedcredits, which will be handled by node A as shown in FIG. 3. Thus, thecredit register of node A will be properly set to the number of creditsthat queue i in node B can hold.

[0028] Note that this mechanism will operate for each queue in node Bthat is used by node A. Further note that this mechanism will operatefor each queue in node A that is used by node B, as each node can bothsend and receive, and thus will have both credits and debits for thedifferent registers.

[0029] Although the present invention and its advantages have beendescribed in detail, it should be understood that various changes,substitutions and alterations can be made herein without departing fromthe spirit and scope of the invention as defined by the appended claims.

What is claimed is:
 1. A method for initializing credit in a creditregister used for flow control that is resident on a first node of amultinode computer system, the method comprising the steps of: loadingan initial credit value into a debit register resident on a second nodeof the multinode computer system; and transferring the initial creditvalue into the credit register by using operational mechanisms of thesystem.
 2. The method of claim 1, wherein: the initial credit value isequal to a size of a queue that is resident on the second node andreceives information from the first node.
 3. The method of claim 1,further comprising the step of: loading the credit register with a zeroprior to the step of transferring.
 4. The method of claim 1, wherein:the step of loading is performed after a reset condition has beenlifted.
 5. The method of claim 1, wherein the step of transferringcomprises the steps of: placing the credit value into a packet which isbound for the first node; decrementing the debit register by the amountof the credit value; and sending the packet to the first node.
 6. Themethod of claim 5, wherein the step of transferring further comprisesthe steps of: receiving, by the first node, the packet from the secondnode; determining whether the packet is destined for the first node;forwarding the packet to another node, if the packet is not destined forthe first node; and processing the packet, if the packet is destined forthe first node.
 7. The method of claim 5, wherein the step oftransferring further comprises the steps of: receiving, by the firstnode, the packet from the second node; inspecting the packet, by thefirst node, to determine whether the packet includes credits for thefirst node; and incrementing the credit register by a value of thecredits determined to be in the packet.
 8. The method of claim 7,further comprising the step of: sending a subsequent packet to thesecond node from the first node.
 9. The method of claim 8, wherein thesecond node includes a queue that receives information from the firstnode, wherein the step of sending the subsequent packet comprises thesteps of: determining an amount of space that the subsequent packet willrequire in the queue of the second node; comparing the amount of spacewith a number of credits in the credit register; sending the subsequentpacket to the second node if the amount of space is less than the numberof credits; and holding the subsequent packet until the amount of spaceis less than the number of credits if the amount of space is not lessthan the number of credits.
 10. The method of claim 9, wherein the stepof sending the subsequent packet further comprises the step of:decrementing the credit register by the amount.
 11. The method of claim9, further comprising the steps of: receiving, by the second node, thesubsequent packet from the first node; inspecting the packet, by thesecond node, to determine whether the subsequent packet includes creditsfor the second node; and incrementing a credit register of the secondnode by the value of the credits determined to be in the subsequentpacket.
 12. The method of claim 9, further comprising the steps of:receiving, by the second node, the subsequent packet from the firstnode; determining whether the subsequent packet is destined for thesecond node; forwarding the subsequent packet to another node, if thesubsequent packet is not destined for the second node; and processingthe subsequent packet, if the subsequent packet is destined for thesecond node.
 13. The method of claim 12, further comprising the stepsof: consuming, by the second node, the subsequent packet; and releasing,by the second node, the amount of credits to the first node.
 14. Amethod for initializing credit in a credit register used for flowcontrol that is resident on a first node of a multinode computer system,the method comprising the steps of: loading an initial credit value intoa debit register resident on a second node of the multinode computersystem; loading the credit register with a zero prior to the step oftransferring; and transferring the initial credit value into the creditregister by using operational mechanisms of the system; wherein theinitial credit value is equal to a size of a queue that is resident onthe second node and receives information from the first node, and thesteps of loading are performed after a reset condition has been lifted.15. The method of claim 14, wherein the step of transferring comprisesthe steps of: placing the credit value into a packet which is bound forthe first node; decrementing the debit register by the amount of thecredit value; and sending the packet to the first node.
 16. The methodof claim 15, wherein the step of transferring further comprises thesteps of: receiving, by the first node, the packet from the second node;inspecting the packet, by the first node, to determine whether thepacket includes credits for the first node; and incrementing the creditregister by a value of the credits determined to be in the packet.
 17. Amechanism for initializing credit in a credit register used for flowcontrol that is resident on a first node of a multinode computer system,the mechanism comprising: means for loading an initial credit value intoa debit register resident on a second node of the multinode computersystem; and means for transferring the initial credit value into thecredit register using operational mechanisms of the system.
 18. Themechanism of claim 17, wherein: the initial credit value is equal to asize of a queue that is resident on the second node and receivesinformation from the first node.
 19. The mechanism of claim 17, furthercomprising: means for loading the credit register with a zero whichoperates prior to the means for transferring.
 20. The mechanism of claim17, wherein: the means for loading operates after a reset condition hasbeen lifted.
 21. The mechanism of claim 17, wherein the means fortransferring comprises: means for placing the credit value into a packetwhich is bound for the first node; means for decrementing the debitregister by the amount of the credit value; and means for sending thepacket to the first node.
 22. The mechanism of claim 21, wherein themeans for transferring further comprises: means for receiving the packetfrom the second node; means for determining whether the packet isdestined for the first node; means for forwarding the packet to anothernode, if the packet is not destined for the first node; and means forprocessing the packet, if the packet is destined for the first node. 23.The mechanism of claim 21, wherein the means for transferring furthercomprises: means for receiving the packet from the second node; meansfor inspecting the packet to determine whether the packet includescredits for the first node; and means for incrementing the creditregister by a value of the credits determined to be in the packet. 24.The mechanism of claim 23, further comprising: means for sending asubsequent packet to the second node from the first node.
 25. Themechanism of claim 24, wherein the second node includes a queue thatreceives information from the first node, wherein the means for sendingthe subsequent packet comprises: means for determining an amount ofspace that the subsequent packet will require in the queue of the secondnode; means for comparing the amount of space with a number of creditsin the credit register; means for sending the subsequent packet to thesecond node if the amount of space is less than the number of credits;and means for holding the subsequent packet until the amount of space isless than the number of credits if the amount of space is not less thanthe number of credits.
 26. The mechanism of claim 25, wherein the meansfor sending the subsequent packet further comprises: means fordecrementing the credit register by the amount.
 27. The mechanism ofclaim 25, further comprising: means for receiving the subsequent packetfrom the first node; means for inspecting the packet to determinewhether the subsequent packet includes credits for the second node; andmeans for incrementing a credit register of the second node by the valueof the credits determined to be in the subsequent packet.
 28. Themechanism of claim 25, further comprising: means for receiving thesubsequent packet from the first node; means for determining whether thesubsequent packet is destined for the second node; means for forwardingthe subsequent packet to another node, if the subsequent packet is notdestined for the second node; and means for processing the subsequentpacket, if the subsequent packet is destined for the second node. 29.The mechanism of claim 28, further comprising: means for consuming, bythe second node, the subsequent packet; and means for releasing, by thesecond node, the amount of credits to the first node.
 30. A mechanismfor initializing credit in a credit register used for flow control thatis resident on a first node of a multinode computer system, themechanism comprising: first means for loading an initial credit valueinto a debit register resident on a second node of the multinodecomputer system; second means for loading the credit register with azero; and means for transferring the initial credit value into thecredit register by using operational mechanisms of the system; whereinthe initial credit value is equal to a size of a queue that is residenton the second node and receives information from the first node, and thefirst and second means for loading operate after a reset condition hasbeen lifted.
 31. The mechanism of claim 30, wherein the means fortransferring comprises: means for placing the credit value into a packetwhich is bound for the first node; means for decrementing the debitregister by the amount of the credit value; and means for sending thepacket to the first node.
 32. The mechanism of claim 31, wherein themeans for transferring further comprises: means for receiving the packetfrom the second node; means for inspecting the packet to determinewhether the packet includes credits for the first node; and means forincrementing the credit register by a value of the credits determined tobe in the packet.